5 research outputs found
Efficient All-to-All Collective Communication Schedules for Direct-Connect Topologies
The all-to-all collective communications primitive is widely used in machine
learning (ML) and high performance computing (HPC) workloads, and optimizing
its performance is of interest to both ML and HPC communities. All-to-all is a
particularly challenging workload that can severely strain the underlying
interconnect bandwidth at scale. This is mainly because of the quadratic
scaling in the number of messages that must be simultaneously serviced combined
with large message sizes. This paper takes a holistic approach to optimize the
performance of all-to-all collective communications on supercomputer-scale
direct-connect interconnects. We address several algorithmic and practical
challenges in developing efficient and bandwidth-optimal all-to-all schedules
for any topology, lowering the schedules to various backends and fabrics that
may or may not expose additional forwarding bandwidth, establishing an upper
bound on all-to-all throughput, and exploring novel topologies that deliver
near-optimal all-to-all performance
Pluralism about Knowledge
In this paper I consider the prospects for pluralism about knowledge, that is, the view that there is a plurality of knowledge relations. After a brief overview of some views that entail a sort of pluralism about knowledge, I focus on a particular kind of knowledge pluralism I call standards pluralism. Put roughly, standards pluralism is the view that one never knows anything simpliciter. Rather, one knows by this-or-that epistemic standard. Because there is a plurality of epistemic standards, there is a plurality of knowledge relations. In §1 I argue that one can construct an impressive case for standards pluralism. In §2 I clarify the relationship between standards pluralism, epistemic contextualism and epistemic relativism. In §3 I argue that standards pluralism faces a serious objection. The gist of the objection is that standards pluralism is incompatible with plausible claims about the normative role of knowledge. In §4 I finish by sketching the form that a standards pluralist response to this objection might take